JMS Data Intake set up - Uniform Distributed Queue - Cluster

Configuration Steps

Pre-requisites for running Data Intake:

  • JMS module will be pointed to a Cycle cluster and every instance in the cluster should be up and running.

  • The property dataIntake.enabled should be set to true on server side of Cycle.

  • Each cycle instance should have its own cycle-coherence-config.xml file and each instance should have a unique port number.

  • Need to provide Customer Name, Intake Profile name , and Class Name (In case of Enrollment) in the loading file, which should be created on OIPA side prior to Data Intake. The steps to be followed on OIPA side can be found in theData Intake Guide.

  • Two new properties are introduced in Cycle.properties files on server side:

    • datatIntake.reProcessingLimit = 10 and datatIntake.yieldTimePeriod = 5000

    • If they are not mentioned/configured - system will use their default values 10 and 5000 respectively.

The following steps are done in the Web Logic Server Console, beginning with the left-hand navigation menu.

Create Cluster:

Environment -> Clusters

  1. Click New tab and provide the name of the cluster and click OK.

Adding Members to the Cluster :

  1. Click the Cluster which is created in previous step.
  2. Click the Severs tab.
  3. Scroll down on the page to see the servers table.
  4. Click the Add button and add the Cycle instances .

Note: Add more than one cycle instance to the cluster.

Create a JMS Server:

Services > Messaging > JMS Servers

  1. Select New
    • Name: DataIntakeJMSServer
    • Persistent Store: (none)
    • Target: CycleCluster
  2. Finish. The JMS server should now be visible in the list with Health OK.

Create a JMS Module:

Services > Messaging > JMS Modules

  1. Select New

    • Name: DataIntakeJMSModule

    • Leave the other options empty

    • Targets: CycleCluster

  2. Click Next

  3. Leave “Would you like to add resources to this JMS system module” unchecked and click Finish.

Create a Sub Deployment:

A subdeployment is not necessary for the JMS queue to work, but it allows you to easily target subcomponents of the JMS module to a single target or group of targets. We will use the subdeployment in this to target the following connection factory and JMS queue to the JMS server we created earlier.

Services > Messaging > JMS Modules

  1. Select DataIntakeJMSModule

  2. Select the Subdeployments tab and New Subdeployment Name: DataIntakeSubdeployment

  3. Click Next

    Select the target(s) for the subdeployment. Choose either Servers (i.e. WebLogic managed servers, such as the DataIntake) or JMS Servers (DataIntakeJMSServer) such as the JMS Server created earlier. As the purpose of subdeployment here is to target a specific JMS server, choose the JMS Server option.

  1. Select the DataIntakeJMSServer created earlier.

  2. Click Finish.

Create a Connection Factory:

Services > Messaging > JMS Modules

  1. Select DataIntakeJMSModule and click New.
  2. Select Connection Factory and click Next.
    • Name: IntakeConnectionFactory
    • JNDI Name: IntakeConnectionFactory
    • Leave the other values at default
    • Targets: CycleCluster
  3. Select Finish.
  4. Click IntakeConnectionFactory - Make sure that Default targeting is enabled.

Note: Uncheck Server Affinity enabled checkbox in Intake Connection factory to maintain uniform distribution of JMS messages across the cluster.

Create a JMS Distributed Queue:

Services > Messaging > JMS Modules

  1. Select DataIntakeJMSModule and click New.
  2. Select DistributedQueue and click Next.
    • Name: DIQueue
    • JNDI Name: DIQueue
    • Template: None
    • Target: Click on Advanced Targeting -> Select DataIntakeSubdeployment
  3. Click Next.
  4. Finish. The JMS queue is now complete and can be accessed using the JNDI names IntakeConnectionFactory and DIQueue.

Data Intake

Work Manager is no longer needed, instead using spring task executor internally. Refer system properties document for newly added data intake properties.

Note: After the above steps are completed, set dataIntake.enabled property to true in cycle Properties file and a server restart is required to pick up data intake configuration.

Change in Data Intake client utility (pas.di.testclient)

  1. Add the below two jar files to pas.di.testclient utility lib folder:
    • wlclient.jar
    • wljmsclient.jar
  2. Modify conf/CycleClient.properties file in pas.di.testclient with the following changes.
  3. Uncomment weblogic context factory and comment other context factories:
    • contextFactory=weblogic.jndi.WLInitialContextFactory
    • Modify provider url
    • provider.url=t3://localhost:7003(provide appropriate url ex: Cycle url)

Each cycle instance should have its own Cycle-coherence-cache-config.xml with unique port numbers and it should be registered as well-known address list of other.

Performance Recommendations

WebLogic Settings

S.No Settings Description
1 Login Timeout =25000 <ManagedServer> -- Configuration -- tuning -- Login Timeout should be 25000
2 HTTP Duration = 60
 <ManagedServer> -- Configuration -- Protocols --HTTP -- Duration should be 60
3 Max Capacity=250
Initial Capacity = 100		 Services --<DataSource> --Configuration -- ConnectionPool -- Maximum Capacity should be 250 and Initial Capacity should be 100
4 GC Algorithm “-XX:+UseG1GC” G1GC to be part of jvm arguments.
5 -Xms4g –Xmx4g Heap memory settings.
6 Uncheck - Server Affinity Enabled Services--Messaging–JMS Modules--DataIntakeJMSModule--IntakeConnectionFactory–Load Balance tab.